Various gaze-controlled interfaces have been proposed for use as input devices for everyday computing or computing devices. Pointing with eye gaze tracking is relatively straightforward, but using eye gaze tracking for performing actions, such as clicks, is challenging. Inaccuracy and inadvertent clicking are the main problems encountered when using eye gaze tracking as an input device for computing devices. The inaccuracy can be caused by gaze tracker error, or it can stem from fundamental limitations such as involuntary eye movements, including jitter and drifts (and the lack of pixel-pointing precision). Inadvertent clicking, also known as the Midas touch problem, occurs because eyes are sensory (input) organs and this creates a mismatch when using them as a means for communicating intention to a computer.
Techniques devised to perform actions (or mimic mouse clicks) on activatable elements or ‘activatables’, such as hyperlinks, buttons, fields or the like for example, are called ‘click alternatives’. Various click alternatives have been proposed for encoding an action with eye gaze. One click alternative is called dwell or fixation, which triggers an action when the gaze dwells on (i.e. fixates) the same area of the display screen for a defined time interval. The dwell click alternative primarily suffers from inadvertent clicking, especially when there are many activatables within a gaze area. To address such problems, improvements to the dwell click alternative can be made by automatically magnifying the area being gazed at if there are multiple activatables. However, magnification brings its own issues such as reducing the content visible on the screen and visual distraction. In addition to this, conscious eye control is generally unnatural and tiring. As a result, click alternatives using blinking, winking or eye gestures are generally not preferred.
Other improved dwell click alternatives use dwell in combination with confirm buttons. This design tries to control inadvertent clicking by providing a confirmation step. In one known form, a single confirm button is presented temporarily to the user when they dwell upon an activatable. The confirm button may be presented near or next to the activatable or alternatively in the margin of the screen. When dwelling on an activatable, the user must then look and dwell on the confirm button to ‘click’ the activatable. In another known form, multiple confirm buttons may be presented temporarily, each confirm button corresponding to a respective activatable in the user's gaze area on screen. Again, the multiple confirm buttons may be presented near or around the gaze area or alternatively in a separate margin to the main display screen. The advantage of multiple confirm buttons over a single confirm button is to account or compensate for the inaccuracy and imprecision of gaze trackers, in a similar way to magnification. When using multiple confirm buttons, the user must be informed which confirm button belongs to which activatable. In one known form, text labels on the confirm buttons match the activatables. In another known form, connection lines are drawn between activatables and the associated confirm buttons. In these forms, the temporary confirm buttons occlude the electronic visual work and change their positions as the gaze moves between activatables.
While reducing inadvertent clicks compared to using dwell alone, the above single and multiple confirm click alternatives can be distracting and tiring for a user. Firstly, such approaches utilise temporarily appearing confirm buttons that appear on the screen content or in a margin when the user is dwelling on an activatable and change as the user dwells on a new activatable or cluster of activatables. The constant appearing and disappearing of confirm buttons as the user gazes in different areas can be very distracting. Additionally, before actuating a confirm button by dwelling on it, the user must check that the confirm button corresponds to the activatable they intended to click via the text labels or connection lines. This can require the user to look back and forward between the intended activatable and the confirm button to check the association, e.g. match a text label on the confirm button with the text of the activatable, or trace the connection lines. This can be extremely tiring and time consuming in the case of the multiple confirm button approach as the user must then traverse the various confirm buttons arbitrarily presented and select the correct one. Additionally, depending on the dwell time thresholds, confirm buttons may change as the user moves their gaze from the activatable area of interest across to the confirm buttons, which may be presented in the margin. If the confirm buttons are presented near the gaze area, they can occlude screen content and cause distraction.
In this specification where reference has been made to patent specifications, other external documents, or other sources of information, this is generally for the purpose of providing a context for discussing the features of the invention. Unless specifically stated otherwise, reference to such external documents is not to be construed as an admission that such documents, or such sources of information, in any jurisdiction, are prior art, or form part of the common general knowledge in the art.
It is an object of the invention to provide an improved gaze-controlled interface method and system to enable a user to interact with electronic visual works, or to at least provide the public with a useful choice.